555 Advanced Online Continuing Legal Education: How to Leverage Technology-Mediated Education --- or Lawyers’ Li --- elong Learning Seth C. Oranburg* Continuing legal education (“CLE”) is theoretically important not only --- or lawyers but also --- or society. CLE aims to keep lawyers’ knowledge o --- ever-changing law and technology up to date so that lawyers can continue advancing law on its long bend toward justice. CLE can socialize lawyers to the rapidly changing human phenomenon we call culture so that they remain keenly aware o --- what justice requires. And CLE could help attorneys --- ul --- ill their duty o --- technological competence by keeping attorneys’ understanding o --- legal technology current. For these reasons, as well as others, most states mandate that lawyers take several hours o --- CLE courses each year. Yet busy lawyers sometimes seek ways to meet mandatory CLE (“MCLE”) required by state bar associations with minimum time and e --- --- ort. It is especially easy to cut corners when taking an online CLE (“OCLE”) course, because attorneys can nominally “watch” online videos while doing billable work. It is hard to imagine how OCLE per --- orms the social purposes contemplated above when attorneys merely watch videos at eight times their recorded speed in marathon sessions simply to meet annual compliance requirements. This article brings learning theory and cognitive science to bear on OCLE pedagogy. In short, learning science shows that engaging in passive learning, like watching videos, is less e --- --- ective --- or learning than active learning, such as engaging with assessments. Moreover, this article identi --- ies how MCLE encourages ine --- --- ective pedagogy by measuring MCLE compliance in terms o --- inputs (hours spent attending lessons) instead o --- outputs (ability to apply in --- ormation learned). MCLE requirements based on time spent watching videos encourage CLE providers to create long videos, whereas learning science --- inds that short videos are more e --- --- ective. MCLE compliance generally does not require completing any assessments, yet learning science shows that *Seth C. Oranburg, Associate Pro --- essor, University o --- New Hampshire Franklin Pierce School o --- Law; Director, Program on Organizations, Business, and Markets, Classical Liberal Institute at NYU School o --- Law; J.D., University o --- Chicago. I thank my research assistants Sammie Lowe and Abby Flynn --- or their substantial support on this project. Journal o --- Legal Education, Volume 72, Number 3 & 4 (2024) 556 Journal o --- Legal Education completing assessments is critical --- or learning. Thus, the problem is that MCLE requirements do not relate to e --- --- ective learning. The solution is not, however, to mandate in-person CLE attendance or to prohibit OCLE --- rom counting toward MCLE. Instead o --- regressing technologically, it is time to advance OCLE by requiring or at least permitting OCLE providers to make OCLE more engaging and e --- --- ective. This article describes how OCLE courses will be more engaging and e --- --- ective by applying “backward design” and “chunking” techniques. Backward design means, --- irst, developing learning objectives; second, conceiving how instructors can assess learners’ achievement o --- these learning objectives; and third, creating video content that prepares students to accomplish the assessment. Chunking means keeping videos short and interactive, interleaving passive learning through video with active learning and questions. This article provides a brie --- how-to guide --- or instructors who want to improve their e --- --- ectiveness quickly yet drastically in teaching OCLE. This article tabulates the costs and bene --- its o --- OCLE, and it --- inds OCLE’s social value likely to be negative. Some would use this evidence in arguments to end our --- i --- ty-year experiment in MCLE requirements, or at least to prohibit OCLE --- rom quali --- ying as MCLE. But this article comes to a di --- --- erent conclusion. OCLE could become --- ar more bene --- icial i --- state bar associations encourage or require OCLE providers to use best practices in technology- mediated education (“TME”). This paper concludes with a call to action, inviting state bar associations to require distance education that --- ul --- ills its pedagogical potential. Distance education is here to stay—accreditors should make sure this power --- ul tool is used e --- --- ectively. Introduction In 1935, law pro --- essor Karl Nickerson Llewellyn delivered a talk at Harvard Law School, “On What Is Wrong with So-Called Legal Education.”1 Lest readers interpret his screed as raving by an ousted member o --- the --- ringe, note that the peer-reviewed Journal o --- Legal Studies identi --- ied Pro --- essor Llewellyn as one o --- the most in --- luential legal scholars o --- all time.2 Llewellyn complained that elite law schools o --- that time did not train lawyers to per --- orm the obligations o --- the legal pro --- ession.3 Rather, law schools taught theory.4 Moreover, law schools were insulated --- rom the normal business necessity to invent and improve or go under.5 While he did not use the modern language o --- law and economics, he explained that law school teaching is a --- unction o --- path 1 Karl Llewellyn, On What is Wrong with So-Called Legal Education, 35 Colum. L. Rev. 651 (1935). 2 Fred R. Shapiro, The Most-Cited Legal Scholars, 29 J. Legal Stud. 409 (2000). According to this study, Llewellyn ranked as the eighteenth-most-cited legal scholar. 3 Lewellyn, supra note 1, at 652–53. 4 Id. at 656. 5 Id. at 667. Advanced Online Continuing Legal Education: How to Leverage 557 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning dependence,6 and that law schools have little incentive to innovate, because they are captured by teachers who enjoy high theory.7 While the Harvard Law Review re --- used to publish Llewellyn’s heretical talk, Columbia Law Review printed it. Llewellyn continued his campaign to re --- orm law schools and is now recognized as --- oundational in bringing the movement known as legal realism to law school education.8 Today, virtually all law schools employ some combination o --- the case law method developed at Harvard9 and more modern applications skills training through clinics and experiential learning10 in the juris doctor curriculum. There appears to be no law school today that relies solely on lectures --- or training lawyers. Yet CLE, especially OCLE, does continue to rely almost solely on lecture modalities --- or teaching. I --- CLE is important enough to be mandatory,11 and i --- OCLE is a valid means o --- meeting MCLE requirements,12 then why do we permit this requirement to be met via ine --- --- ective teaching methods? The answer to this question is partly the same as what Llewellyn expressed almost ninety years ago: Some combination o --- path dependence and capture results in in --- erior OCLE. Yet the --- ixes to the J.D. curriculum and the OCLE 6 Id. at 652–53. 7 Id. at 665–66. 8 See William Twining, Karl Llewellyn and the Realist Movement 27–28 (2d ed. 2012). Twining credits Llewellyn’s mentor, Arthur Linton Corbin, along with Wesley Newcomb Hoh --- eld and Walter Wheeler Cook, with leading the --- irst phase o --- the legal realist movement. Corbin, Newcomb, and Cook rebelled against the then-established “Yale system,” which taught law entirely by lectures. Llewellyn, Corbin’s protégé, joined the Yale --- aculty in 1903 and set about “to work out a method o --- teaching and o --- exposition o --- legal doctrine which overcame the inadequacies o --- the Yale system.” 9 Christopher C. Langell introduced the case law method to American legal education in 1870. See Bruce A. Kimball, The Proli --- eration o --- the Case Method Teaching in American Law Schools: Mr. Langdell’s Emblematic “Abomination,” 1890–1915 (2007). Kimball describes the case law method as “regnant” and an “ultimate triumph” in law schools today while recognizing it was initially derided as an “abomination.” See also Anthony Chase, The Birth o --- the Modern Law School, 23 Am. J. Legal Hist. 329 (1979) (pinpointing the start o --- the modern law school teaching method as occurring on the opening day o --- the --- all term in 1870 at Harvard Law School in Dane Hall, where Pro --- essor Langdell taught a class based on a pamphlet that he would later call his “casebook” on contracts). 10 While recognizing that law school vastly increased skills training in the modern era, Erwin Chemerinsky, then Dean o --- Univeristy o --- Cali --- ornia, Berkeley, School o --- Law, somewhat dismally concludes that law school has not changed much in the past --- orty years, and it will not change much in the next --- orty. Erwin Chemerinsky, Re --- lections on the Future o --- Legal Education, 13 FIU L. Rev. 215, 215 (2018) (“But overall, legal education today is much like what it was in 1975 when I was a IL.”). Id. at 227 (“My conclusion is that, --- orty years --- rom now law schools probably will look much like the law schools o --- today.”). 11 The --- act that most ( --- orty-nine out o --- --- i --- ty-six) U.S. jurisdictions have MCLE requirements suggests that most states think CLE is important enough to mandate. 12 All states that have MCLE requirements permit at least some, i --- not all, o --- these requirements to be met via OCLE courses. 558 Journal o --- Legal Education curriculum are not necessarily the same. It may be impracticable and, in --- act, undesirable to use the case law method in OCLE. The audience is di --- --- erent, and the asynchronous modality cannot --- ully mimic the classroom. But this does not result in the inevitable conclusion that online OCLE cannot be improved. Rather, we can apply assessment-based learning to OCLE to reinvigorate what has become a vestigial anachronism. I. What’s Wrong with So-Called Online Continuing Legal Education The big problem with OCLE is that states do not require or even credit OCLE providers --- or using online teaching technologies that include some o --- the highlights o --- this modality, such as assessment, personalization, and gami --- ication. States generally count MCLE in terms o --- hours o --- content watched or attended, but this is not a good measure o --- learning, especially online, where distractions are plenti --- ul and course videos can be sped up. Since OCLE rules are basically MCLE rules, providers have no incentive and, arguably, little --- lexibility to motivate adult learning. So, instead o --- --- ixing problems with CLE, OCLE pixelates them, making its problems harder to resolve with lower resolution. Since OCLE is predominately an online projection o --- in-person CLE, and since so much more is written about CLE generally than about OCLE in particular, we might consider whether OCLE meets goals designed --- or it by evaluating goals o --- CLE, and, more generally, what the objectives o --- adult continuing education are. What learning objectives should OCLE meet, and what does it accomplish? Pedagogically, we now understand that humans are li --- elong learners.13 Malcolm Shepherd Knowles began advancing the idea o --- adult education around 1950, when he published his guide --- or administrators, leaders, and teachers.14 By 1999, li --- elong learning (LLL) became commonly used in educational conversations as the leading organizing principle o --- education.15 Learning science clearly recognizes the increasingly important need to continue education and acquire new skills, especially regarding in --- ormation technology.16 Lawyers increasingly recognize how CLE could --- acilitate technical competence in particular.17 13 Roberta L. Duy --- --- , The Value o --- Li --- elong Learning: Key Element in Pro --- essional Career Development, 99 J. Am. Diet Assoc. 538–43 (1999). 14 Malcolm S. Knowles, In --- ormal Adult Education: A Guide --- or Administrators, Leaders, and Teachers (1950). 15 Marjan Laal, et al., Continuing education; li --- elong learning, 116 Soc. & Behav. Sci. 4052 (2013). 16 Michal Beller & Ehud Or, The Crossroads between Li --- elong Learning and In --- ormation Technology: A Challenge Facing Leading Universities, 4 J. Computer-Mediated Comm. (2006), http://jcmc. indiana.edu/vol4/issue2/beller.html. 17 The contributions by John G. Browning and Danielle M. Hall in this issue o --- the Journal o --- Legal Education support this proposition. Advanced Online Continuing Legal Education: How to Leverage 559 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning On the assumption that continuing legal education provides current in --- ormation that all attorneys need to know,18 states began making CLE mandatory in the 1970s, seemingly in response to public opinion o --- attorney incompetence and corruption that was echoed by a 1973 speech by Chie --- Justice Warren Burger.19 Within two years, two states, Iowa and Minnesota, adopted MCLE requirements.20 Today, --- orty-nine out o --- --- i --- ty-six American jurisdictions require MCLE.21 All these jurisdictions permit at least hal --- o --- MCLE to be obtained through OCLE. While attorney competence is surely a worthy goal, does OCLE produce attorney competence? I --- so, what are the tangible bene --- its o --- that competence, and what does it cost? Despite vast interdisciplinary literature on LLL and pro --- essional competence, there is almost zero evidence that MCLE accomplishes its stated goal o --- increasing attorney competence.22 The District o --- Columbia studied CLE --- or two years and published a nearly 200-page report that concluded there is no empirical data to demonstrate that CLE courses improve attorney competence.23 Even states that require CLE acknowledge a lack o --- empirical evidence showing any correlation between CLE and attorney competence.24 Meanwhile, courts have --- ound that --- ailing to meet CLE requirements does not establish incompetence.25 There is likewise zero empirical evidence that OCLE makes attorneys more competent. 18 Lisa A. Grigg, The Mandatory Continuing Legal Education (MCLE) Debate: Is it Improving Lawyer Competence or Just Busy Work?, 12 BYU J. Pub. L. 417, 426 (1998). 19 See generally Warren E. Burger, The Special Skills o --- Advocacy: Are Specialized Training and Certi --- ication o --- Advocates Essential to Our System o --- Justice?, 42 Fordham L. Rev. 227 (1973) (expressing “anxieties . . . concerning the quality o --- advocacy in our courts”). 20 Grigg, supra note 18, at 418. 21 See Attachment A (“CLE Requirements in U.S. Jurisdictions”). 22 One empirical study reports a statistical correlation between increases in requiring CLE and decreases in docketed cases regarding attorney sanctions. Chris Ziegler & Justin Kuhn, Is MCLE a Good Thing? An Inquiry into MCLE and Attorney Discipline, Continuing Legal Educ. Reguls. Ass’n, https://www.clereg.org/assets/pd --- /Is_MCLE_A_Good_Thing.pd --- . Notably, however, that scholarship was sponsored by the Continuing Legal Regulators Association, and its --- indings do not appear in any peer-reviewed publication. That paper did not per --- orm multiple-regression analysis and so it could not conclude whether this correlation is spurious. 23 Evaluation and Recommendations on MCLE --- or The District O --- Columbia Bar (1995). 24 Cali --- ornia State Bar Commission’s Final Report o --- the Commission to Study Mandatory Continuing Legal Education, at 7 (1988) (the report “acknowledge[s] the lack o --- any statistical evidence clearly demonstrating a direct, positive correlation between MCLE and attorney competence”). 25 People v. Ngo, 14 Cal. 4th 30, 36 (1996) (“Although the right to counsel clearly entails a right to competent representation by a licensed attorney, and although MCLE requirements clearly do relate to pro --- essional competence, in the sense they are intended to enhance the competence o --- attorneys practicing law in this state, the in --- erence is unwarranted that 560 Journal o --- Legal Education Attorneys clearly need to learn new skills, especially technological competencies, during their decades-long careers. Yet CLE cannot prove it is up to this task. And OCLE is hamstrung by timebound requirements. OCLE is literally just online CLE, whereas technology enables online teaching to be much more than a hologram o --- a classroom. What change in rules could motivate advanced OCLE? Can we unlock the power o --- TME to --- oster LLL --- or lawyers? II. Advanced OCLE Advanced OCLE means OCLE courses that use teaching tools unique or speci --- ic to technology-mediated education. Quizzes, polls, collaborative whiteboards, asynchronous discussion boards, and games are examples o --- TME interactive learning tools well known since the 1990s.26 Perhaps there exist missed opportunities to use TME and the most popular online teaching organization principle, backward design, in LLL --- or lawyers. A. Backward Design Online learning must be designed to account --- or the bene --- its and drawbacks o --- the online learning environment.27 Online courses can be student-guided, where students can spend more time on concepts with which they need more help and less time on concepts they pick up quickly.28 Online learning is more e --- --- icient --- or motivated sel --- -regulated learners,29 while students who lack independence and sel --- -motivation have lower success rates in online courses than their peers.30 Student engagement is a multidimensional concept, however, and teachers can increase student engagement through e --- --- ective course design.31 any and all noncompliance with those requirements necessarily establishes an attorney’s pro --- essional incompetence or constitutionally de --- icient per --- ormance in representation --- ollowing enrollment on inactive status.”). 26 Christopher J. Dede, The Evolution o --- Distance Learning: Technology-Mediated Interactive Learning, 22 J. Res. Computing Educ. 247–64 (1990). 27 See, e.g., Jared Keengwe & Terry T. Kidd, Towards Best Practices in Online Learning and Teaching in Higher Education, 6 MERLOT J. Online Learning & Teaching 533 (2010); Rob Abel, Implementing Best Practices in Online Learning, 28 Educause R. 1528 (2005); Mary Rose Grant & Heather R. Thornton, Best Practices in Undergraduate Adult-Centered Online Learning: Mechanisms --- or Course Design and Delivery, 3 MERLOT J. Online Learning & Teaching 246 (2007); Henry S. Merrill, Best Practices --- or Online Facilitation, 14 Adult Learning 13 (2006). 28 Lisa Kirtman, Online Versus In-Class Courses: An Examination o --- Di --- --- erences in Learning Outcomes, 18 Issues in Teacher Educ. 103–16 (2009), http://search.proquest.com/ docview/233320851?accountid=27700. 29 Ji Won You & Myunghee Kang, The role o --- academic emotions in the relationship between perceived academic control and sel --- -regulated learning in online learning, 77 Computers & Educ. 125–33 (2014). 30 Wilhelmina C. Savenye, Improving Online Courses: What is Interaction and Why Use It?, 2 Distance Learning 22–28 (2005). 31 Fatima A --- zal & Lynn Craw --- ord, Student’s Perception o --- Engagement in Online Project Management Advanced Online Continuing Legal Education: How to Leverage 561 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning Wiggins and McTighe provided teachers with what is now one o --- the most well-established online course design strategies and called it backward design.32 Backward design (1) identi --- ies desired results --- or students called “learning objectives,” (2) determines acceptance evidence that students achieved these learning objectives through “assessments,” and (3) plans learning experiences that help students achieve these assessments. Backward design means, practically, that course builders start with learning objectives.33 It means to know where you’re going so that you better understand where you are now so that the steps you take are always in the right direction.34 There --- ore, backward design begins with mapping learning objectives to the topics and themes o --- the course. With well-de --- ined targets on a learning map, instructors can chart courses that get students there—but --- rom where? Course maps must account --- or both where students begin and where instructors hope they end. Assessing student knowledge at the beginning o --- the course de --- ines the starting point. Then, course builders connect the students’ starting point to the learning end goal with knowledge o --- the learning terrain and wisdom --- rom having guided many students through it. Goals are motivating. When objectives are paired with assessments that provide --- eedback on learning, students can measure progress toward goals, quickly note shortcomings, and engage with personalized learning activities that help them achieve criteria; students become more motivated to participate in learning because they see how that learning bene --- its them. Alenezi explained how backward design --- osters student motivation:35 Education and Its Impact on Per --- ormance: The Mediating Role o --- Sel --- -Motivation, 3 Project Leadership & Society 1 (2022). 32 See, e.g., Grant Wiggins & Jay McTighe, What Is Backward Design?, in Ryan S. Bowen, Understanding by Design (2005). 33 See, e.g., Jan Emory, Understanding Backward Design to Strengthen Curricular Models, 39 Nurse Educator 122 (2014). 34 See Stephen R. Covey, et al., The Seven Habits o --- Highly E --- --- ective People (2020) (“Begin With the End in Mind means to start with a clear understanding o --- your destination. You need to know where you are going in order to better understand where you are now so that the steps you take are always in the right direction.”). 35 Huda Alenezi, Learning as the Prize: Enhancing Students’ Intrinsic Motivation through Backward Design, 23 Int’l J. Pedagogy & Curriculum 1 (2015). 562 Journal o --- Legal Education Table 1. Alenezi’s Model o --- the Role o --- Backward Design to Foster Students’ Intrinsic Motivation to Learn. The role o --- backward design to Students who are intrinsically --- oster students’ motivation to learn motivated . . . ... Begins with the end to determine Learn based on their learning needs students’ needs Focuses on enduring Learn relevant and meaning --- ul understandings and what is worth knowledge teaching Achieve true understanding Teaches concepts in depth Appreciate what they learn Relates goals to students’ lives Promotes knowledge and skills that Apply what they learn students will use Have sel --- -con --- idence and realize Allows students to experience errors their own success and achievement to improve their learning Determine their strengths, Plans ongoing assessment using the weaknesses, and misconceptions six --- acets o --- understanding Know where they are headed, work Considers the WHERETO toward notable goals and engage in elements when planning learning learning activities Note that the backward design process begins, not ends, with de --- ining objectives. A --- ter de --- ining objectives, instructors should plan assessments so students can determine whether they are meeting those learning objectives. De --- ining meaning --- ul objectives, then giving students visibility into whether are accomplishing objectives, motivates students to continue learning. Backward design gives students visibility into the purpose o --- the instruction process and thus motivates students to engage with course materials.36 MCLE requirements, however, are at odds with this pedagogical strategy. Instead o --- requiring assessments, which would --- it with backward design, MCLE encourages lectures. For an example o --- a typical MCLE requirement, consider Cali --- ornia’s standard --- or being an eligible MCLE activity provider --- or a single 36 Viorica Condrat, Backward Design: When a Good Ending Makes a Good Beginning, IV International Spring Symposium Proceedings 64-75 (2018); Junho Son, The E --- --- ect o --- Backward Design Re --- lecting Process-Focused Assessment on Science Learning Achievement and Science Learning Motivation o --- Elementary School Students, J. o --- Korean Soc’y o --- Earth Science Educ. 90–106 (2018); Camille Chesley & Tarida Anantachai, Level Up the One-Shot: Empowering Students with Backward Design and Game- Based Learning, in Motivating Students on a Time Budget: Pedagogical Frames and Lesson Plans --- or In-Person and Online In --- ormation Literacy Instruction (Sarah Steiner & Miriam Rigby eds., 2019). Advanced Online Continuing Legal Education: How to Leverage 563 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning activity37 or multiple activities.38 In both cases, the provider must provide a syllabus, handouts, and/or a PowerPoint presentation.39 Assessments as in- class activities are not contemplated by the rule.40 While Cali --- ornia MCLE requires providers to --- urnish a syllabus discussing the content o --- a course,41 there is no requirement to de --- ine learning outcomes. Cali --- ornia is hardly unique in requiring CLE providers to --- urnish evidence o --- in --- ormation covered, but learning theory shows that in --- ormation covered is not necessarily in --- ormation learned. In --- act, learning theory suggests that online learning is e --- --- ective only where the online course is structured to take advantage o --- the unique pedagogical bene --- its o --- TME. MCLE requirements like Cali --- ornia’s treat OCLE just like CLE, and CLE encourages traditional lecture-style courses. OCLE should be regulated di --- --- erently than in-person CLE because online courses work di --- --- erently. Learning scientists David George Glance, Martin Forsey, and Myles Riley summarized the pedagogical --- oundation o --- online courses whose characteristics have well-established pedagogical bene --- its. These are the distinguishing --- eatures o --- good online courses and those --- eatures associated with bene --- its --- or learning.42 Table 2. Characteristics o --- MOOCs and Related Pedagogical Bene --- its. Credit Glance, Forsey, and Riley (2013). Online Course Characteristic Pedagogical Bene --- its Online mode o --- delivery E --- --- icacy o --- online learning Online quizzes and assessments Retrieval learning Short videos and quizzes Mastery learning Peer and sel --- -assessment Enhanced learning through this assessment Short videos Enhanced attention and --- ocus Online --- orums Peer assistance, out–o --- –band learning The common attribute o --- all these bene --- icial characteristics o --- online courses is an emphasis on engagement with the learner through online technologies. 37 Becoming a Cali --- ornia Single Activity Provider, State Bar o --- Cali --- ornia, https://www.calbar. ca.gov/Attorneys/MCLE-CLE/MCLE-Providers/Single-Activity-Providers. 38 Id. 39 Id. 40 Id. 41 Id. 42 David George Glance, et al., The Pedagogical Foundations o --- Massive Open Online Courses, 18 First Monday (2013). 564 Journal o --- Legal Education The asynchronous environment becomes a --- eature, not a bug. Instead o --- merely displaying long videos, success --- ul online courses regularly engage students with TME. While most state bar associations permit OCLE courses to --- ul --- ill MCLE requirements, these same states treat OCLE the same as CLE through rules that encourage in --- ormative lectures. State bar rules that impose the same requirements on OCLE as in-person CLE prevent advanced OCLE innovations that are likely to make OCLE more e --- --- ective, such as by applying backward design. To unlock the potential o --- better learning through technology, states should change OCLE rules to permit i --- not to require backward design with assessment-based learning in OCLE courses. B. Advanced OCLE Rules Current MCLE requirements, exempli --- ied by Cali --- ornia’s rules discussed above, generally credit OCLE based on time spent in class. This article has explained why this method results in ine --- --- icient online learning. Given that spent spend watching online lectures in unrelated to how much students learn, how else could state bar associations credit OCLE i --- not by time? Perhaps state bar associations could take note o --- how law schools are adapting to online learning, also known as distance education. The American Bar Association (“ABA”) standard --- or distance education in the J.D. curriculum not only credit students --- or time spent on assessments, it also a --- --- irmatively requires engagement strategies in distance learning courses. ABA Standard 306. Distance Education. (2) [E]ach Distance Education Course [must] include[] at least two o --- the --- ollowing: (i) providing direct instruction; (ii) assessing or providing --- eedback on a student’s coursework; (iii) providing in --- ormation or responding to questions about the content o --- a course; or (iv) --- acilitating a group discussion regarding the content o --- a course. State bar associations could likewise require that OCLE courses --- eature some engaging TME. But simply adopting ABA Standard 306 is not enough to advance OCLE in most jurisdictions. Bar associations also need to credit students (and instructors) --- or time spent on assessments. Once again, the ABA provides a model rule: ABA Standard 310. Determination o --- Credit Hours --- or Coursework. Advanced Online Continuing Legal Education: How to Leverage 565 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning … (b) A “credit hour” is an amount o --- work that reasonably approximates: (1) not less than one hour o --- classroom or direct --- aculty instruction and two hours o --- out-o --- -class student work per week --- or --- i --- teen weeks, or the equivalent amount o --- work over a di --- --- erent amount o --- time. ABA Standard 310 echoes the U.S. Department o --- Education de --- inition o --- “credit hours.”43 Both use the pedagogical term o --- art “direct --- aculty instruction” (“DFI”). DFI re --- ers to a wide variety o --- activities students expect in classrooms,44 including:45 • Reading articles and writing summaries about them • Writing blog posts and journal entries • Analyzing case studies and discussing practice problems with instructors or classmates • Attending con --- erence calls and videocon --- erences • Participating in verbal debates • Posting and reading posts on discussion boards and other asynchronous --- orums • Going on --- ield trips and virtual --- ield trips • Preparing and presenting papers and presentations • Collaborating in group projects • Researching guided projects • Taking online quizzes • Reviewing automated and personalized --- eedback • Assessing peer work • Re --- lecting on learned concepts orally or in writing • Playing learning games State bar associations could adopt similar language --- or MCLE hours. State bar associations should expressly permit advanced OCLE by crediting time to TME activities. 43 34 CFR § 600.2. 44 See Direct Instruction, The Glossary o --- Education Re --- orm (2013), https://www.edglossary. org/direct-instruction/#:~:text=In%20general%20usage%2C%20the%20term%20 direct%20instruction%20re --- ers,teachers%2C%20such%20as%20in%20a%20lecture%20 or%20demonstration. 45 See Credit Hour Policy, Rosemont College, https://www.rosemont.edu/about/credit-hour- policy.pd --- ; Direct Faculty Instruction in Online Courses, UMass Lowell, https:// --- aculty.gps.uml. edu/wp-content/uploads/2021/09/Direct-Instruction-in-Online-Courses.pd --- . 566 Journal o --- Legal Education C. Advanced OCLE Is Backward Provided that states adopt rules permitting OCLE to credit outputs (evidence o --- learning) and not just inputs (time spent receiving in --- ormation), OCLE providers could use backward design to advance OCLE by using TME to make learning more customized. Here is an example o --- how an OCLE course using backward design could be structured. 1. Learning Objectives Backward design suggests that course builders start with learning objectives.46 To use a popular expression, attributed to Zig Ziglar, “You can’t hit a target you cannot see, and you cannot see a target you do not have.” Likewise, Tony Robbins allegedly said, “You can’t hit a target i --- you don’t know what it is.” These quotations are meant to emphasize the importance o --- establishing goals. I --- you do not have a goal --- or a course, then learning in it is basically random, and assessments seem (and perhaps are) irrelevant to learning. Planning a course without --- irst establishing learning objectives is like going on a journey without a map or a destination. When you take your --- irst step, have you gotten closer to or --- arther --- rom your goal? Without a destination, this is impossible to de --- ine. Backward design principles, there --- ore, suggest establishing learning objectives --- irst. Learning objectives thus --- unction metaphorically, like the destination at which your students should arrive upon completing the course. An ancillary bene --- it o --- establishing learning objectives --- or all online CLE courses is that CLE students can select courses based on what they want to learn. Some people want to learn about a topic because o --- general interest, while others want to develop skills that empower them to do certain things. CLE can meet a wide variety o --- learning objectives, but it is important to de --- ine these objectives so adult learners can select courses based on what they value learning. Here are some examples o --- learning objectives that would be appropriate --- or online CLE courses: • A --- ter this course, you will be able to identi --- y whether a transaction is a “hybrid transaction” under the 2022 Amendments to the Uni --- orm Commercial Code. • A --- ter this course, you will remember the top three most common cybersecurity mistakes lawyers make when storing con --- idential client in --- ormation, and you will be able to apply strategies to avoid these mistakes. • A --- ter this course, you will be able to use the HotDocs automated document assembly so --- tware to generate a dra --- t lease agreement --- or a client. 46 See, e.g., Emory, supra note 33, at 122. Advanced Online Continuing Legal Education: How to Leverage 567 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning Note that all these objectives are speci --- ic (well de --- ined), measurable (assessable), assignable (completable in a short course), realistic (connected to pro --- essional lawyering), and timebound (measured at a speci --- ic time, usually at the end o --- the course). This is called the SMART system --- or de --- ining objectives,47 and it is a notably e --- --- ective way to communicate learning objectives to students.48 The SMART method is commonly stated as the standard --- or developing e --- --- ective, measurable goals and objectives.49 With established objectives, the instructor can then begin establishing waypoints. These waypoints are assessments. Assessments tell the instructor and the students where learners are and direct learners to where they must go to achieve goals. 2. Assessment The ABA recognizes three ways o --- engaging law students in distance education in addition to providing video lectures. In --- act, the ABA requires that all distance education courses that count toward a juris doctor degree have at least one o --- these three elements: assessment, response to questions, and group discussion. O --- these elements, only one seems easily applicable to online CLE: assessment. This article recommends that OCLE providers include pre-tests and post-tests in OCLE courses. Moreover, this article recommends that state bar associations credit time spent on these tests so that a one-hour MCLE requirement could be met with a shorter video plus some assessments. Group discussions and responses to questions both require time and instructor availability. These are hard to achieve in a one-hour online course, especially when that course is asynchronous and on-demand. I --- a student chooses to take a course at three in the morning on a Sunday, it may be in --- easible --- or an instructor to be standing by to answer questions that hour, and it may be unlikely that other students are available --- or discussion at that point. However, assessment can be programmed and automated, so an always- on computer responds to students with --- eedback at any time o --- the day or night. Since assessment is the only one o --- these engagement strategies --- easible --- or online CLE, this article will --- ocus on adopting and requiring assessment instead. This is not a radical or even a new idea. In 1968, three leading educational psychologists wrote an in --- luential book explaining that the most important 47 The SMART objectives paradigm is usually ascribed to management scholar George T. Doran, There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives, 70 Management R. 35 (1981). 48 Debnath Chatterjee & Janet Corral, How to Write Well-De --- ined Learning Objectives, 19 J. Educ. Perioperative Med. 610 (2017). 49 May Britt Bjerke & Ralph Renger, Being Smart about Writing SMART Objectives, 61 Eval. & Program Planning 125–27 (2017). 568 Journal o --- Legal Education --- actor in --- luencing learning is assessment.50 Now, the notion that assessment is central to learning is so central to the --- ield o --- learning science that entire books are written about the key role o --- assessment --- or durable learning.51 It is virtually impossible to --- ind current university resources on e --- --- ective teaching that do not discuss assessment.52 What, then, is assessment, and why is it important --- or learning? The University o --- Massachusetts Amherst O --- --- ice o --- Academic Planning & Assessment de --- ines assessment as “the systematic collection and analysis o --- in --- ormation to improve student learning.”53 This is more than just testing via --- inal exams; the de --- inition encompasses more than just evaluating students. The evaluative --- unction o --- assessment, which is an assessment o --- learning, is called summative assessment. Summative assessment cans certi --- y achievement and quali --- y someone to practice.54 The bar exam is an example o --- a summative assessment—and anyone who took that grueling test is likely grate --- ul never to take such a stress --- ul examination again. Assessment can also be used --- or learning, and this is called --- ormative assessment. Formative assessments, such as diagnostics, pre-tests, and check- ins, play important roles in motivating learning.55 A quiz given be --- ore a lesson learned not only tells the teacher what the students do not know (and there --- ore what they should learn), but it also tells the students what they can expect to learn.56 I --- a course o --- --- ers a pre-test be --- ore the lesson, then gives the lesson, then gives the post-test, students can use this testing to establish what they should learn, become more receptive to learning that in --- ormation, and demonstrate to themselves and others that they learned it. Moreover, according to the testing 50 David Paul Ausubel, et al, Educational Psychology: A cognitive View (2d ed. 1978). 51 Peter C. Brown, et al., Make It Stick: The Science o --- Success --- ul Learning (2014); Benedict Carey, How We Learn: The Surprising Truth about When, Where, and Why It Happens (2015); Pooja K. Agarwal & Patrice M. Bain, Power --- ul Teaching: Unleash the Science o --- Learning (2019); Yara Weinstein, et al. , Understanding How We Learn: A Visual Guide (2018). 52 See, e.g., Michael R. Fisher Jr., Student Assessment in Teaching and Learning, Vanderbilit University Center --- or Teaching, https://c --- t.vanderbilt.edu/student-assessment-in-teaching-and- learning/#:~:text=Student%20assessment%20enables%20instructors%20to,ine --- --- ective%20 ones%20in%20their%20pedagogy. 53 Martha L.A. Stassen, et al., Program-Based Review and Assessment: Tools and Techniques --- or Program Improvement (2011), at 5, https://www.umass.edu/oapa/sites/ de --- ault/ --- iles/pd --- /handbooks/program_assessment_handbook.pd --- ; see also Martha L.A. Stassen, et al., Course-Based Review and Assessment: Methods --- or Understanding Student Learning (2011), https://www.umass.edu/oapa/sites/de --- ault/ --- iles/pd --- / handbooks/course_based_assessment_handbook.pd --- . 54 Lancaster University Educational Development, Principles and Purposes o --- Assessment in Higher Education (2021), https://www.lancaster.ac.uk/ curriculum-and-education-development-academy/resources/assessment-principles/. 55 Id. 56 Id. Advanced Online Continuing Legal Education: How to Leverage 569 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning principle, this use o --- assessment promotes long-term memory o --- lessons learned.57 And assessment can be used as learning, meaning the assessment can itsel --- be the lesson. This is called sustainable assessment.58 For example, students who are asked to judge the quality o --- their work per --- orm two valuable learning --- unctions.59 First, i --- given proper tools --- or sel --- -assessment, students are steered to --- ocus on what is important to learn. Second, students learn and develop metacognitive abilities that “sustain” their learning by helping them accrue tools that help them learn more critically.60 It may not surprise most readers that empirical analysis o --- attitudes regarding assessment shows that teachers like assessments more than students do.61 But this is at least partly because students o --- ten think o --- assessments as high- stakes, time-pressured, white-knuckled testing experiences that distinguish who merits some credential --- rom who does not. This accurately describes summative assessments such as the bar exam. It would be unsurprising to learn that online CLE participants, who generally choose which courses they take, would choose not to go through this rigorous summative assessment process when they have less stress --- ul options. But high-stakes summative assessments are just one type o --- assessment o --- learning. Assesements o --- learning can be low-stakes, too. For example, an instructor can give learners multiple opportunities to take an assessment. Assessment so --- tware can display to learners which questions they got right and wrong, and why, so learners can review speci --- ic lessons be --- ore attempting the assessment again. Management studies show that low-stakes assessment increases student engagement and student retention.62 For this reason, this article suggests that OCLE providers adopt gentler, lower-stakes assessments, especially when the online learning environment makes low-stakes assessments so easy to provide and per --- orm. For example, one e --- --- ective way to add assessment to an OCLE course is to o --- --- er a pre-test and a post-test. A pre-test literally means giving students a test be --- ore they learn the material. This may seem counterintuitive, --- or how can teachers expect students to ace tests be --- ore they learn the lessons? But 57 Make it Stick, supra note 51. 58 Lancaster, supra note 54. 59 Student Sel --- -Assessment: Re --- raming Assessment as Learning, Rice University Center --- or Teaching Excellence (2019), https://cte.rice.edu/blog/2019/student-sel --- -assessment. 60 Lancaster, supra note 54. 61 Richard B. Fletcher, et al, Faculty and Students Conceptions o --- Assessment in Higher Education, 64 High. Educ. 119 (2011). 62 Nicky M. Meer & Amanda Chapman, Assessment --- or Con --- idence: Exploring the Impact that Low-Stakes Assessment Design Has on Student Retention, 12 Int’l J. Management Educ. 186, 191 (2014) (“This research has demonstrated that an early low-stakes assessment strategy addresses student concerns and anxieties around their abilities.”). 570 Journal o --- Legal Education learning science shows that pre-tests have two bene --- icial --- eatures. First, the pre-test establishes what students already know so teachers can begin at the appropriate learning level. Second, the pre-test primes students to learn new in --- ormation.63 Another reason students dislike assessments is that they seem irrelevant to learning.64 This is because some teachers use assessments poorly, not because assessment is inherently worthless. I --- you have ever taken a test that does not seem to resemble what you learned in class, you have experienced the assessment relevancy gap. This may occur because instructors --- eel obligated to give tests but --- ail to understand how assessment is part and parcel o --- learning. To bridge the assessment relevancy gap, the course must be designed with understanding in mind.65 The “understanding by design” --- ramework established by Jay McTighe and Grant Wiggins accords with a larger literature recommending that online courses be built using “backward design” strategies. The third and --- inal step in creating OCLE via backward design is developing learning activities relevant to the assessments and their objectives. When the elements o --- the course are connected in this way, students are more likely to perceive the relevance o --- the learning activities, which helps motivate them to --- ocus on those activities. The pre-test method, in particular, primes students to pay attention to videos, because students then watch those videos not just --- or completion credit but to obtain in --- ormation they will need later. 3. Activity “Chunks” Instructors create e --- --- ective learning activities by breaking in --- ormation into chunks that are each associated with some assessment. There are many examples o --- learning activities that range --- rom lectures to games.66 Games- based learning is especially e --- --- ective because pedagogical games can be quick bites o --- --- un, motivating, personalized, and experiential learning activity.67 63 Pre-testing relies on the so-called testing e --- --- ect, which is a strong and well-demonstrated phenomenon that learners’ retention is more durable when they are tested on what they learn. There is some debate about whether pre-tests or post-tests are more e --- --- ective --- or retention, but there seems to be consensus that both are help --- ul --- or retention and that pre- and post-tests work in tandem to improve learning and retention. This bene --- it is in addition to the bene --- its associated with teaching the students --- rom the appropriate baseline and motivating students by showing them how they learned --- rom the course. Alice Latimier, et al., Does Pre-testing Promote Better Retention than Post-testing?, 4 npj Sci. Learn. 15 (2019), https:// doi.org/10.1038/s41539-019-0053-1; see also John Gough, What’s the Big Deal about Pre-Tests?, 27 Prime Number 3–7 (2012). 64 Fletcher et al., supra note 61. 65 Wiggins McTighe, supra note 32. 66 Teaching and Learning: Examples o --- Learning Activities, University o --- Tasmania, https://www.teach- ing-learning.utas.edu.au/learning-activities-and-delivery-modes/planning-learning-activities/ examples-o --- -learning-activities. 67 Stephen Tang, et al., Introduction to Games-Based Learning (2009). Advanced Online Continuing Legal Education: How to Leverage 571 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning This article submits that MCLE rule changes could permit MCLE providers to o --- --- er games instead o --- lectures; perhaps some industry providing adult learning games --- or lawyers would emerge to cater to the next generation o --- lawyers who grew up gaming. But while gami --- ication is one e --- --- ective strategy to improve OCLE learning, it is not the closest at hand. Rather, this article tries to take a more realistic approach by suggesting how lecture videos—currently the dominant --- ormat o --- OCLE—could be made into more e --- --- ective learning activities using backward design, as discussed above, and “chunking” principles. For at least --- i --- ty years, cognitive scientists generally agreed that learning is organized by chunking.68 Chunking theory is based on an understanding o --- how memory works. Adult learners have short-term memory, working memory, and long-term memory. Learning works best when students can process some set o --- in --- ormation within working memory. (Retrieval practice later helps commit that knowledge to long-term memory.) Lessons thus need to be broken down into working-memory-sized chunks. There is some academic debate on how big a chunk is,69 but most learning theory suggests that one chunk o --- learning is smaller than most teachers imagine. Instead o --- getting into the entire learning theory o --- chunking, this article will apply what it expects will be the most common --- orm o --- learning activity in OCLE: lecture videos. How long a video is too long? A 2021 study concluded that the optimum length --- or an online lecture on engineering topics is six to ten minutes.70 The Vanderbilt Center --- or Teaching likewise recommends online videos o --- less than six minutes each.71 One disadvantage o --- online lectures is that it is harder --- or learners to passively watch online lectures than in-person ones.72 And even in person, adult learners struggle to --- ocus on lectures exceeding eighteen minutes.73 68 Herman Buschke, Learning Is Organized by Chunking, 15 J. Verbal Learning & Verbal Behavior 313–24 (1976). 69 See, e.g., Herbert A. Simon, How Big Is a Chunk?, 183 Science 482–88 (1974). 70 Amhad Manasrah, et al., Short Videos, or Long Videos? A Study on the Ideal Video Length in Online Learning, 2021 Int’l Con --- erence on In --- o. Tech. 366–470 (2021). 71 Cynthia J. Brame, E --- --- ective Educational Videos: Principles and Guidelines --- or Maximizing Student Learning --- rom Video Content, 15 CBE–Li --- e Sciences Educ. 1–6 (2016). 72 Tom Cherrett, et al., Making training more cognitively e --- --- ective: Making videos interactive, 40 British J. Educational Tech. 1124–34 (2009), http://dx.doi.org/10.1111/j.1467-8535.2009.00985.x; Itiel Dror, et al., A cognitive perspective on technology enhanced learning in medical training: Great opportunities, pit --- alls and challenges, 33 Medical Teacher 291–96 (2011), http://dx.doi.org/10.3109/014215 9X.2011.550970; Juho Kim, et al., Understanding in-video dropouts and interaction peaks in online lecture videos, at 31–40, http://dx.doi.org/10.1145/2556325.2566237. 73 Neil A. Bradbury, Attention span during lectures: 8 seconds, 10 Minutes, or More?, 40 Advances Physiology Educ. 509–13 (2016) (although this paper suggests that mixing up material can engage students during in-person class periods o --- longer than eighteen minutes, it admits 572 Journal o --- Legal Education Yet most OCLE --- eatures hours-long content. There is no evidence or theory that suggests videos this long are pedagogically optimal. Instead, MCLE is generally required in hour-long denominations, which seems to be why OCLE videos are one hour long. Why, then, are OCLE courses essentially required to be three to ten times too long? Whatever the reason may be, it does not appear rooted in pedagogy. Fortunately, students can attend longer lessons at which instructors employ periodic assessments to maintain engagement. Multiple peer-reviewed studies show that adding interactivity to videos substantially increases participation rates.74 OCLE courses could pause every --- ive to ten minutes to ask --- or a learner response, which could in turn result in a di --- --- erent instructional video played next. Yet most online CLE courses entirely lack the engaging characteristics that make online learning e --- --- ective. Instead, online CLE courses are generally little more than low- --- idelity recordings o --- live lectures. It seems unsurprising that they are ine --- --- ective, as they are designed without these well-established pedagogical requirements o --- online learning in mind. This section explained how advanced OCLE could take advantage o --- TME and thus become more engaging, more e --- --- ective, and more --- un. It also identi --- ied what MCLE rules state bars need to change to enable, --- acilitate, and require advanced OCLE. By essentially adopting ABA rules pertaining to distance education in the J.D. curriculum—such as crediting coursework based on more than the length o --- videos watched and requiring some online engagement such as knowledge check-in—state bar associations could inspire advanced OCLE that is pedagogically more e --- --- ective than in-person CLE instead o --- requiring basic OCLE that must be a less e --- --- ective projection o --- an in-person course. III. Analysis: Is Advanced OCLE Worth It? One well-established way to evaluate policies and regulations is through cost-bene --- it analysis.75 As state bar associations and OCLE providers consider implementing the advanced OCLE regulations and best practices suggestions above, they might consider the costs and bene --- its o --- the status quo. Currently, most states’ MCLE requirements do not distinguish between CLE and OCLE. There appears to be no theory suggesting that status quo OCLE, which consists mainly o --- noninteractive long- --- orm videos, which in turn are o --- ten just low- --- idelity recordings o --- lectures, should produce e --- --- ective the consensus that most students’ attention spans last about ten minutes). 74 Nitza Geri, et al., Challenging the six-minute myth o --- online video lectures: Can interactivity expand the attention span o --- learners?, 5 Online J. Applied Knowledge Mgmt. 101 (2017). 75 Jean Drèze & Nicholas Stern, The Theory o --- Cost-Bene --- it Analysis, in Handbook o --- Public Economics 909–89 (1987); Tev --- ik F. Nas, Cost-Bene --- it Analysis : Theory and Application (2d. ed. 2016); Jacques Lesourne, Jacques Cost-Bene --- it Analysis and Economic Theory (1975). Advanced Online Continuing Legal Education: How to Leverage 573 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning learning. While one might presume that attorneys generally learn something --- rom watching OCLE videos passively, learning theory suggests that it is an ine --- --- icient way to learn that is unlikely to result in long-term retention o --- in --- ormation. Empirically, as discussed in Part I, supra, there appears to be no evidence that OCLE causes attorneys to become more competent. Anecdotally, consider your own experience meeting MCLE requirements by watching OCLE videos at the eleventh hour. Most attorneys reading this article have some personal experience blipping through an online CLE video to meet some last-minute compliance requirement. When you think o --- the most meaning --- ul learning experiences o --- your pro --- essional li --- e, how many basic OCLE courses come to mind? Theory, evidence, and anecdote all suggest that bene --- its --- rom basic OCLE are low. What, then, are its costs? Mandating CLE produces direct costs (the costs o --- attending these sessions) and indirect costs (the costs o --- not doing something else --- or that time). A 2022 ABA survey --- ound there are about 1.3 million lawyers in the United States.76 Our independent research estimated 1,302,091 licensed lawyers in 2023. Moreover, our research estimated the number o --- attorneys in each o --- --- i --- ty-six U.S. jurisdictions. We then determined the MCLE hours requirements in each o --- those jurisdictions and established that U.S. attorneys must spend about 13 million hours each year on MCLE. The cost o --- 13 million hours o --- MCLE is hard to determine precisely, as CLE costs range widely. As a low estimate, we might assume that attorneys meet all their CLE requirements through a low-cost provider o --- unlimited CLE courses like Quimbee, which charges $179 per year per attorney,77 resulting in just over $233 million annually spent on CLE courses. I --- we instead assume that most lawyers pay ABA rates --- or OCLE webinars, with those courses averaging $120 per hour, the total MCLE spending soars to over $1.5 billion annually. Reality is probably somewhere between, with some attorneys selecting the lowest-cost OCLE providers and others choosing premium in-person CLE providers. The direct cost is likely at least a quarter-billion dollars annually. Then there is the indirect cost o --- attorneys’ time. To calculate this, we researched the average billable hour rate in all U.S. jurisdictions that have MCLE requirements. We multiplied the number o --- attorneys in each jurisdiction by the hours o --- MCLE they must accrue annually and by the average hourly billable rate in that jurisdiction. This reveals an opportunity 76 ABA survey --- inds 1.3M lawyers in the U.S., American Bar Association (June 20, 2022), https:// www.americanbar.org/news/abanews/aba-news-archives/2022/06/aba-lawyers-survey/. 77 O nline Continuing Legal Education (CLE) & MCLE Courses, Quimbee, h t t p s : / / w w w. g o o g l e a d s e r v i c e s . c o m / p a g e a d / a c l k ? s a = L & a i = D C h c S E w i - 0 t z - C i _ q E A xV 1 X k c B H R Fw B A kYA B A B G g J x d Q & a e = 2 & g c l i d = E A I a I Q o b C h M I - v t L c w o v 6 h A M Vd V 5 H A R 0 R c A Q J E A AYA i A A E g J 6 O v D _ B w E & o h o s t = w w w. google.com&cid=CAASJeRoPHg_3SbJeL6LccvDD3 --- 0VHD7e0dbPrXJzWolMDIAH- dxU6bg&sig=AOD64_3M --- chZrQ CBdR8jvUAH9OsxB_xdRQ&q&adurl&ved=2a- hUKEwiimtXCi_qEAxUEF1kFHUb9BogQ0Qx6BAgMEAE&nis=2&dct=1. 574 Journal o --- Legal Education cost o --- over $3.6 billion in time spent on MCLE that could ostensibly be spent on billable hours. These calculations do not even consider the instructor’s time, which could be spent teaching more e --- --- ectively otherwise, or the producer’s time, which could at least theoretically be spent making more e --- --- ective learning products. Thus, CLE should bear the burden o --- proving that it o --- --- ers about $4 billion annually in educational value in exchange --- or its costs. MCLE costs at least $4 billion annually. Yet we --- ound no evidence that MCLE, in general, and the OCLE option to meet that requirement, in particular, serve society. There is zero evidence that OCLE is pedagogically e --- --- ective. This cost-bene --- it analysis is not a close call. Rather, these back-o --- - the-envelope calculations suggest that OCLE is in dire need o --- substantial overhauls. Some may argue that the way to --- ix this imbalanced cost-bene --- it equation is to remove the requirement --- or MCLE or limit how many OCLE hours count as MCLE credits. Some states have taken this approach and eliminated CLE requirements accordingly. Yet another approach is to raise the bene --- its o --- online CLE by bringing it up to date with modern pedagogy. Instead o --- using a pure lecture --- ormat o --- education debunked and discarded one hundred years ago by law schools, online CLE could be made e --- --- ective through the modern pedagogical strategy o --- assessment-based learning. Conclusions Some readers who see the tabulation o --- the costs and bene --- its o --- MCLE may conclude that MCLE does not o --- --- er bene --- its that merit its costs, so it should not be mandated. Eliminating MCLE is, admittedly, one solution to the problem that OCLE does not appear to provide bene --- its worth what it costs, at least --- rom a cost-bene --- it standpoint. But the cost-bene --- it analysis provided here is just a starting point that does not paint the whole picture. For one thing, it is unlikely that the costs and bene --- its o --- CLE are evenly distributed. Further research may show that large, wealthy law --- irms bear the highest costs o --- MCLE, while attorneys serving indigent populations gain the most bene --- its. I --- MCLE has some redistributive e --- --- ect on society, there could be policy reasons to maintain the mandate. Moreover, it is too early to give up on OCLE. Advanced OCLE could provide greater bene --- its at lower costs by using TME to make li --- elong learning more accessible, --- un, customized, and impact --- ul. To get advanced OCLE, however, state bar associations must change rules by distinguishing in-person and distance learning based on how those di --- --- erent modalities have di --- --- erent bene --- its and drawbacks. Basic OCLE magni --- ies the problems with conventional CLE lecture --- ormats. While it may be hard to stay --- ocused in a lecture hall --- or an hour, it is much harder to maintain that --- ocus on videos online. Online learning Advanced Online Continuing Legal Education: How to Leverage 575 Technology-Mediated Education --- or Lawyers’ Li --- elong Learning could o --- --- er huge bene --- its by providing attorneys access to high-quality education whenever and wherever they want, yet studies show that students simply have a hard time paying attention to videos.78 Basic OCLE, consisting o --- taped lectures, is e --- --- ectively nothing more than a poorer version o --- an in- person experience. Instead o --- OCLE being a mere hologram or projection o --- traditional CLE, a move to advanced OCLE could make distance continuing education uniquely valuable. This article takes the position that remote learning can work well when done right. This article thus runs counter to reports in the popular press, such as The Wall Street Journal article titled “The Results Are In --- or Remote Learning: It Didn’t Work.”79 Many people share the sentiment that distance education was a --- ailed pandemic experiment, but the data simply does not support that conclusion, at least not regarding higher education. Studies o --- higher education showed that, even during the emergency circumstances o --- the COVID-19 pandemic, online education worked --- or adult learners.80 In --- act, a recent meta-study showed no evidence that o --- --- line education works better regarding medical education.81 In --- act, the data shows that online learning can work, especially --- or adult learners. But online learning must be designed to account --- or the bene --- its and drawbacks o --- the online learning environment.82 Moreover, the strategies --- or success --- ul online learning are not especially new or disputed. Rather, best practices in online teaching and learning have been well established --- or at least two decades. They are not complicated, nor are they disputed. The essential 78 Rahul RK. Rahul, et al., Real Time Attention Span Tracking in Online Education, 11 Int’l J. Innovative Tech. & Exploring Engineering 11 (2022) (discussing how online learning students try to bypass lessons entirely just to receive the accreditation --- or completing the course). 79 Tawnell D. Hobbs & Lee Hawkins, The Results Are In --- or Remote Learning: It Didn’t Work, Wall Street Journal (June 5, 2020), https://www.wsj.com/articles/schools-coronavirus-remote- learning-lockdown-tech-11591375078 (“This spring, America took an involuntary crash course in remote learning. With the school year now winding down, the grade --- rom students, teachers, parents and administrators is already in: It was a --- ailure.”). 80 Meixun Zheng, et al., Online Learning During COVID-19 Produced Equivalent or Better Student Course Per --- ormance as Compared with Pracademic: Empirical Evidence --- rom a School-wide Comparative Study, 21 BMC Medical Education 495 (2021) (“the online cohort during summer quarter 2020 was equally or more likely to get an A course grade than the analogous --- ace-to- --- ace cohort during summer quarter 2019.”). 81 Leisi Pei & Hongbin Wu, Does Online Learning Work Better than O --- --- line Learning in Undergraduate Medical Education? A Systemic Review and Meta-Analysis, 24 Med. Educ. Online (2019) (“Although not all o --- the included research studies reported that using online learning methods in medical education was more e --- --- ective than o --- --- line learning, none o --- the included studies concluded that online learning was less e --- --- ective than o --- --- line methods, regardless o --- the statistical method used. We need to recognize that online learning has its own advantages --- or enhancing students’ learning and should be considered a potential teaching method in medical education.”). 82 See, e.g., Keengwe & Kidd, supra note 27; Abel, supra note 27; Grant & Thornton, supra note 27; Merrill, supra note 27. 576 Journal o --- Legal Education point --- or present purposes is that online learning must include more than watching videos. Why does OCLE --- ail to --- ollow the obviously well-agreed-upon best practices in online learning? One reason OCLE courses may not con --- orm with best practices in online education is that states do not require it. Perhaps, like the colleagues Llewellyn complained about in the 1930s, who grew com --- ortable teaching ine --- --- ectively, our CLE providers are likewise com --- ortable earning substantial pro --- its without demonstrating their courses are related to learning. It is high time to disrupt that path-dependent equilibrium by introducing new rules that enable and require advanced OCLE. State bar associations can take cues --- rom the ABA, whose law school accreditation standards --- or J.D. programs establish standards that enable and require best practices --- or online J.D. courses. State bar associations, which accredit OCLE providers, should take a page --- rom the ABA’s accreditation book and require standards --- or OCLE that are like what the ABA requires --- or online J.D. courses. This could open possibilities --- or a new era o --- advanced OCLE that uses TME to make li --- elong learning meaning --- ul, e --- --- icient, e --- --- ective, and --- un.